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Description 



Method and circuit arrangement for picture-in-picture 
insertion 

5 

BACKGROUND OF THE INVENTION 
1. Field of the Invention 

10 The invention relates to a method for picture-in- 
picture insertion, in which a sequence of decimated 
inset pictures is written to a memory with at least two 
segments and is read out for insertion into a sequence 
of main pictures, to be precise in accordance with the 

15 preamble of claim 1, and also a circuit arrangement for 
picture-in-picture insertion, in particular for 
carrying out said method in accordance with the 
preamble of claim 6. 

2 0 2. Background Art 

Various methods and apparatuses for inserting one or 
more inset pictures into a main picture (PIP - Picture 
In Picture) are known. In this case, the inset pictures 

25 stored in the memory are read out synchronously with a 
main picture. Since the read-out speed is generally 
higher, in a manner corresponding to the decimation of 
the inset picture, than the write-in speed, a seam can 
occur in the inset picture on account of the write 

30 pointer being overtaken by the read pointer, since the 
inset picture is then composed of a current part and a 
preceding part. 

In particular in the case where the two parts originate 
35 from different motion phases, a disturbing effect is 
produced since moving objects through which the seam 
passes are displayed in a distorted manner. Moreover, 
if the frequencies of the inset and main pictures do 
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not correspond exactly, the seam drifts, which is 
perceived as particularly unpleasant. 

EP 0 739 130 A2 discloses, for the purpose of avoiding 
5 this problem, storing two inset pictures (or fields in 
each case) in the memory device, so that it is always 
the case that exactly one picture can be read out while 
the next picture is written. Although this prevents the 
write pointer from being overtaken by the read pointer, 
10 there is nonetheless a significant disadvantage in that 
the storage capacity must be very high, which is 
associated with considerable costs. 

BRIEF SUMMARY OF THE INVENTION 

15 

The invention is based on the object, therefore, of 
providing a method and also a circuit arrangement for 
picture-in-picture insertion of the type mentioned in 
the introduction by means of which, with relatively low 
20 outlay, an inset picture can be generated without a 
seam. 

This object is achieved by means of a method which is 
distinguished by the fact that: 
25 - the inset pictures are written to the memory 
device in a circulating manner under continuously 
incremented write addresses, 

the first address of each written-in inset picture 
is stored, 

30 - an overtake signal is formed by comparing the 
instantaneous address with a previously stored address, 
said signal indicating whether a previous address has 
been reached again and, consequently, the corresponding 
picture content has been overwritten, 

35 - by evaluation of the overtake signal, the current 
or preceding segment is selected for read-out depending 
on whether or not overtaking took place before the 
start of the read-out, and 
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the inset picture stored in the selected segment 
is read out with continuously incremented read 
addresses and is inserted into the main picture. 

5 The object is furthermore achieved by means of a 
circuit arrangement for inserting a sequence of 
decimated inset pictures into a sequence of main 
pictures, which has a memory device having at least two 
segments for the inset pictures, a write controller and 

10 a read controller and which is distinguished in 
particular by the fact that an overtake signal that can 
be fed into a display controller can be generated by 
the write controller, in that a segment buffer is 
provided, which segment buffer is connected to the read 

15 controller and serves to store a first and a last 
address of an inset picture, and in that a memory 
segment to be read out by the read controller can be 
selected by means of the display controller in a manner 
dependent on the overtake signal. 

20 

The solutions according to the invention are based on 
the insight that the write pointer can be prevented 
from being overtaken by the read pointer through 
suitable circulating addressing of the memory device in 

25 a manner utilizing the speed difference between the 
pointers. A significant advantage of this solution is 
that this holds true even when the size of the inset 
picture changes. Furthermore, is also not necessary to 
provide completely separate memory areas for reading 

30 and writing or to adapt the organization of the memory 
to the inset-picture size. 

The subclaims contain advantageous developments of the 
invention . 

35 

Accordingly, for example, the write and read addresses 
are continuously incremented from a first memory 
address up to a last memory address and are in each 
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case reset to the first memory address again after the 
last memory address has been reached. 

For this purpose, in the circuit arrangement according 
5 to the invention, it is preferably provided that the 
write controller and the read controller each have an 
address counter for incrementing the write addresses 
and read addresses, respectively. 

10 BRIEF DESCRIPTION OF THE DRAWINGS 

Further details, features and advantages of the 
invention emerge from the following description of 
preferred embodiments with reference to the drawings, 
15 in which: 



Figure 1 shows a block diagram of a circuit 
arrangement according to the invention; 

20 Figure 2 shows a diagram of the signal profiles when 
writing to a picture memory; 

Figure 3 shows a diagram of the signal profiles when 
reading from a picture memory; 

25 

Figure 4 shows a representation for illustrating the 
origination of a seam; 

Figure 5 shows a diagram of the signal profiles 
30 according to the invention; 

Figure 6 shows a block diagram of a circuit 
arrangement for selecting a memory segment; 

35 Figure 7 shows an address pointer representation in 
the absence of vertical decimation; 



Figure 8 



shows a representation of memory division in 
the absence of vertical decimation, and 
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Figure 9 shows a representation of the memory outlay 
in the case of decimation. 

5 DETAILED DESCRIPTION OF THE INVENTION AND PREFERRED 
EMBODIMENTS THEREOF 

In accordance with Figure 1, a main picture signal is 
fed to a circuit arrangement according to the invention 

10 via a first input A, and an inset picture signal is fed 
via a second input B. The inset picture signal passes 
to a decimation device 1 and also to a synchronization 
device 8. The picture signal of the inset picture, 
which is generally vertically decimated, is buffer- 

15 stored in the form of fields in a memory 2 and, after 
read-out, is fed together with the main picture signal 
to an insertion apparatus 3, which generates a 
composite picture signal. 

20 The synchronization device 8 is connected to the 
decimation device 1 via a first output and to a write 
controller 4 via a second output. A first output of the 
write controller 4 is connected to the memory 2, while 
a second output is connected to a segment buffer 7 and 

25 a third output is connected to a display controller 6. 
The output of the segment buffer 7 is connected to a 
read controller 5, to which a first output of the 
display controller 6 is also fed. An output of the read 
controller 5 is connected to the memory 2. Finally, a 

30 second output of the display controller is fed to the 
insertion apparatus 3, the display controller having an 
input C for synchronization signals of the main 
picture . 

35 In order to write to the memory 2 an inset picture 
signal that has been decimated by the decimation device 
1, the write controller 4 generates the required 
addresses. In the simplest case, given an e.g. linear 
organization of the memory 2, the write address pointer 
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is incremented after each write operation. If the 
address pointer reaches the last address, it is reset 
to the first address, under which the writing process 
is continued, this being done over the duration of the 
5 field with interruptions in accordance with the 
decimation. In this case, a preceding picture (field) 
is inevitably overwritten. 

With each beginning of a new field, the first write 
10 address is stored in the segment buffer 7, which 
additionally stores the last address as well. 
Furthermore, the field position and size are also 
stored in the form of lines and pixels per line in the 
segment buffer 7, the data respectively stored last not 
15 being overwritten. The storage capacity of the segment 
buffer 7 is thus about double said data to be stored 
for a field, which in each case corresponds to a field 
in an instantaneous segment and a current segment in 
the memory 2 . 

20 

Each time the write pointer is incremented, the write 
controller 4 additionally checks whether the new 
address has a specific offset, with respect to the 
previously stored (not with respect to the currently 

25 stored) address. This offset may also be zero in the 
case of a high vertical decimation factor. It serves 
essentially as a safety margin in order to avoid 
overtaking as a result of asynchronous data acceptance 
or slightly deviating vertical frequencies. With this 

30 offset, an overtake signal is generated which indicates 
whether the address has been reached again and hence 
this picture content has been overwritten. 

The content of the segment buffer 7 is transferred to 
35 the read controller 5, by means of which the memory 2 
is read beginning at one of the two addresses present 
after the start of reading at the insertion position, 
so that the inset picture can be generated in 
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accordance with its position and size in the main 
picture by the insertion apparatus 3. 

On the basis of the overtake signal transmitted by the 
5 write controller 4, the display controller 6 decides 
which of the two segments present in the memory 2 is 
read out. The selection of the segment initially always 
proceeds from the last segment. If overtaking took 
place before the start of reading, then the 
10 instantaneous (current) segment is read out. A 
corresponding segment selection signal is transmitted 
to the read controller 5. 

The insertion position is also calculated by means of 
15 the display controller. The display controller 6 
furthermore serves for correcting the raster positions, 
this being effected by comparison between the raster 
position of a displayed picture and the raster position 
of a stored picture and skipping or repetition of a 
20 line. Finally, the display controller 6 can also be 
utilized for multi-picture insertion. 

Before the algorithm is described in detail, firstly 
the terms used in this context will be explained. 

25 

As was explained above, the decimated pixels of the 
inset picture are firstly stored in the memory 2. For 
display purposes, they are read from the memory again 
in the temporal framework of the main picture. For this 
30 purpose, the write and read addresses have to be 
generated by separate address counters. 

Figure 2 shows the corresponding signal profiles when 
writing to the memory 2. The instantaneous value of the 
35 write address counter represents a write pointer with a 
decimated inset pixel in the memory 2. The terms "write 
pointer" and "read pointer" shall be used for this 
reason . 
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An acquisition window (vaqwin) begins with a pulse 
(vaqstart) , and the write address pointer rises with 
each decimated pixel. Since only the visible part of 
the inset picture is decimated, the write address 
5 pointer rests whenever there is no valid line available 
from the vertical decimation stage. After the last line 
to be decimated, the pointer is reset to the start 
position. The picture of a ramp with shoulders results 
in the representation. 

10 

Figure 3 shows the corresponding signal profiles when 
reading from the memory 2. The instant at which a start 
signal (vdisstart) must be generated in order to begin 
reading from the memory 2 is calculated by means of the 

15 display controller 6 on the basis of the desired 
insertion position of the decimated inset picture in 
the main picture. With each line of the main picture, 
at the corresponding insertion position (hdisstart), a 
line of the decimated inset picture is then read from 

20 the memory and inserted into the main picture (video 
mux) . 

The picture of a ramp with shoulders once again results 
in the representation of the read address pointer. The 
25 ramps of the read and write address pointers differ in 
respect of their average gradient. The latter will be 
considered in detail below since it represents the rate 
at which a number of memory cells are swept over within 
a specific period of time. 

30 

The speed of the write address pointer changes with the 
decimation factor, to be precise in such a way that the 
ramp of the write address pointer becomes less steep as 
the decimation factor rises. 

35 

Figure 4 shows the origination of a seam for the case 
of representation in the frame mode. If only a field 
memory is present and the inset picture has been 
decimated, the write pointer is generally overtaken by 
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the read pointer. Since the sources for the inset 
picture ("insert_f ield") and the main picture 
("parent_f ield") are generally asynchronous with 
respect to one another, a seam is thus produced. In 
5 Figure 4, this is indicated by the point of 
intersection between the relatively slow write pointer 
and the read pointer that overtakes the latter, the 
upper-case letters A, B designating the raster position 
of the inset picture and the lower-case Greek letters 
10 tSt/ $6 designating the raster position of the main 
picture . 

From a temporal standpoint, the current field is read 
before the seam, while part of an older picture is 
15 reproduced after the seam. It should be noted in this 
case that, as a result of the line interlacing method 
(interlaced mode), the raster position changes in the 
event of overtaking, and this subsequently has to be 
corrected again. 

20 

The seam becomes very clearly visible in particular 
when a moving picture is displayed, that is to say when 
the current picture and the older picture contain 
different motion phases. If different standards are 

25 used for the main picture and the inset picture, then 
rolling of the seam occurs. As a result of different 
frame frequencies, the combination of the raster 
positions at the beginning of the representation 
changes a number of times every second. It is a very 

30 complicated procedure to perform a correction in a 
manner dependent on this incorrect position. Moreover, 
the picture can be very erratic in the vertical 
direction and be perceived as unpleasant. 

35 The method according to the invention and the circuit 
arrangement according to the invention now allow, in 
particular in the frame mode, a seamless representation 
of pictures that have been decimated to different 
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extents, without two separate field memories having to 
be available. 

For the explanations below, it will be assumed that 
5 orthogonal memory division has been chosen. As a 
result, the memory is divided into lines with fixed 
start addresses. The length of such a line in the 
memory is determined by the largest picture. 

10 Furthermore, it will be assumed that the inset picture 
and the main picture are present in the same standard. 
The two picture sources can be asynchronous but, in 
terms of their time frame, should initially have no 
deviations from one another. If the inset picture is 

15 not decimated vertically and horizontally, one field 
memory suffices for generating a seamless picture-in- 
picture representation, since the two address pointers 
cannot overtake one another on account of their 
identical speed. The case of raster position correction 

20 forms an exception. This correction is performed in the 
first line given a corresponding combination of main 
and inset picture positions. 

In this case, the read address pointer jumps by one 
25 line. In the course of this jump, the situation where 
the write address pointer is overtaken must be 
prevented. This is achieved in that there is space for 
two additional lines in the memory. 

30 Furthermore, suitable control of the memory accesses is 
necessary. The memory is written to with the fields in 
a circulating manner. As a result, with each new field, 
the start address is shifted by the number of 
additionally present lines in the direction of lower 

35 physical addresses. If the physical end address of the 
memory is reached in the course of writing, then a jump 
is made back to the start address. 
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Figure 5 shows the corresponding signal profiles during 
write control (a) and read control (b) . Accordingly, 
there is thus always somewhat more than one inset field 
in the memory. In a similar manner to the case of the 
5 organization of two field memories, in this case, too, 
the write address pointer determines the memory segment 
enable. The signals "vaqstart", "vaqwin", "vdisstart" 
and "hdisstart" again have the same meaning as in 
Figures 2 and 3 . 

10 

Figure 6 shows a block diagram for selecting the 
respectively valid memory segment. The circuit 
comprises an address counter 12, whose output is 
connected to a first register 10 and to a first input 

15 of a comparator 14, a second register 11, to whose 
input the output of the first register 10 is connected, 
and also a multiplexer 13, whose first input 
(line_adr_next ) is connected to the output of the first 
register 10 and whose second output is connected to the 

20 output of the second register 11. This latter output is 
also connected to a second input of the comparator 14, 
whose output is connected to a flip-flop 15. 

For control of the read accesses, the start addresses 
25 of the old and new inset fields are stored in the 
registers 10, 11. With the beginning of a new field, 
the older of the two register contents is rejected, and 
the formerly new start address becomes the old address, 
while the present current address becomes the new start 
30 address. If the beginning of the older inset field has 
been overwritten as a result of the memory being 
written to in a circulating manner, it can no longer be 
read. 

35 For selection of the valid memory segment, the content 
of the address counter 12 (line_address ) is continually 
compared with the start address of the older of the two 
fields (line_address_cur) in the comparator 14. In the 
event of correspondence, the flip-flop 15 is set and 
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the start address of the new field is then present at 
the output of the multiplexer 13. If a new field is 
begun, then the flip-flop is reset, and, as a result of 
the change of the register contents, the same start 
5 address as before is present at the output of the 
multiplexer 13, until this is also overwritten again. 
In this way, the memory enable points to a valid memory 
segment at every instant. The memory space that is 
additionally present means that the read pointer cannot 
10 reach or overtake the write pointer even in the event 
of a jump on account of the raster position correction. 

In order to satisfy the general requirements, however, 
this sequence must be extended. Considerable deviations 

15 from the standard can occur particularly in the case of 
video recorders which are operated with fast forward or 
rewind with picture reproduction. In this case, by way 
of example, it is also necessary to take account of the 
maintenance state and the wear of the tape material. 

20 The memory control must be able to compensate for the 
effects of a stretched tape and also synchronism 
fluctuations of the drive mechanism. However, a 
precondition in this case is that the sync pulse 
separation still operates correctly in the case of such 

25 a signal. 

The vertical frequency f v and the horizontal frequency 
f H are related through the number Z of lines as 
follows : 

30 

fv = f h / Z (5.1) 

The line frequencies of the main picture (fHp) and of 
the inset picture (f H i) are of interest for the 
35 algorithm. Their fluctuations directly affect the 
writing and reading speed. The larger the line 
frequency f H i of the inset picture, the more memory 
content is written per unit time. The smaller the line 
frequency f Hp of the main picture, the fewer lines are 
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If the same standard is 
5 following relationships 
desired line frequency: 
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opposite correspondingly holds 

used in both sources, then the 
hold true, where f H is the 



fHi_max = f H (1 + df Hi ) (5.2) 

fHi_min = f H (1 + df Hi ) (5.3) 

10 f H p_max = f H (1 + df Hp ) (5.4) 

fHp_min = f H (1 + df Hp ) (5.5) 



It will additionally be assumed that no vertical 
picture decimation is performed. Since the sources are 

15 again two sources that are asynchronous with respect to 
one another, write and read pointers can adopt any 
desired position with respect to one another. Equally, 
the pointers can overtake one another in both 
directions given corresponding combination of the 

20 horizontal frequencies. 



Figure 7 shows the memory lines required for the 

respective write and read pointers, where Z acq is the 

number of picture lines of a field that are used for 
25 acquisition. The representation makes it clear how many 

lines must additionally be present in the memory in 

order to prevent overtaking of the pointers in both 
directions . 



30 Z s2 = Z acq { ( 

f Hi max f Hp min) / ^Hi max } (5.6) 
2 S 1 = 2 acq { ( f Hp_max ~ f Hi_min) /f Hp_max } (5.7) 



The total amount of additionally required memory for 
the seamless picture-in-picture representation turns 
35 out to be: 



Zg — 2 acq + Z s x + Z S 2 



(5.8) 
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During writing, the memory contains part of an old 
field and part of a new field. Through the position of 
the write address pointer, one of the two fields is 
enabled for reading. If fewer than Z sl lines of the new 
5 field have been written, the old field is enabled. 
Otherwise, the distance from the beginning of the old 
field is less than Z S 2 lines, with the result that the 
new field can be read. 

10 Figure 8 shows the memory division in the case where no 
vertical decimation is performed. 

If decimation is then effected in the horizontal and 
vertical direction, this influences the speed of the 
15 write address pointer. The corresponding relationships 
are shown in Figure 9. The rise of the ramp is smaller 
in this figure. 

It will initially be assumed that the writing speed 
20 changes to a considerable extent as a result of the 
decimation. By contrast, the fluctuations due to 
changes in the line frequency shall be small. 

It follows from this that the read pointer can no 
25 longer be overtaken by the write pointer on account of 
the speed difference. As a result, the memory outlay 
for seamless picture-in-picture representation can now 
be determined using the decimation factors and the 
fluctuation range of the picture sources . 

30 

2sl_dec (deC ver ) = ( Z aC q/deC ver ) (l - fHi_mindeC ver /fHp_max) 

(5.9) 

The additional requirement of lines decreases as the 
35 decimation factor increases. The maximum emerges for a 
vertical decimation factor dec ver of 1 (vertically 
undecimated picture) . 
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The possibility of the read pointer being overtaken by 
the write pointer will now also be taken into account. 
For small vertical decimation factors , the result is a 
further additional memory requirement of: 

5 

2 s 2_dec = (Zacq/deC V er) (l"fHp_min deC ver /f Hi_max) (5.10) 

For a meaningful result, the expression in the right- 
hand brackets must be positive. The validity of this 
10 expression is thus limited to a range of: 

"1" less than equal to "dec ver " less than equal to 

" ( f Hi_raax/ f Hp_min ) " (5.11) 

15 For values which are greater than the right-hand limit, 
Z S 2_dec shall be set to zero. 

Taking account of the validity ranges, the additional 
memory requirement results from the sum of Z s i dec and 
2 0 Z s2 _dec- The examinations made at the beginning are a 
special case for a vertical decimation factor of 1. 



The total memory requirement consequently turns out to 
be : 

Z g (dec ver ) = Z acq /dec V er + Z s i_ dec + Z S 2_dec for "1" less than 
equal to "dec ver " less than equal to "f H i_max/fH P _min) " 

(5.12) 



30 thus resulting in the following: 



Z g (dec ver ) - Z acq /de Wer w L Hp min deC ver / J- Hi max 

f Hi_min/ ( deC V er f Hp_max) (5.14) 



35 Otherwise, the following holds true: 
Z g (deCver) = Z acq /dec V er + Z s i_ dec for 

"dec ver " greater than " ( f H i_max/fH P _min) " (5.13) 
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The following results from this: 

Zg(deCver) = Z aC q/deC ver (2 - fHi_min/ (deC ver fHp_max) 

(5.15) 

The total number of memory cells required has its 
maximum for a vertical decimation factor of 1 . As the 
decimation factor increases, the memory cell 
requirement greatly decreases. 

It shall also supplementarily be pointed out that when 
the memory 2 is extended to three segments, the method 
according to the invention can also be employed with a 
frame frequency of 100 Hz in the AABB raster. 



